-
Notifications
You must be signed in to change notification settings - Fork 30
Compile time path #101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Compile time path #101
Conversation
Introduces `jsonpath-rust-impl` crate as a procedural macro for JSONPath parsing and validation. Includes AST definitions, Pest-based grammar, and integration with the main library. Starts refining error handling and debugging within parsing logic.
…AP once we know what the spec should be
Introduce `parse_terminated_nonempty` to enforce nonempty parsing for `PestIgnoredPunctuated`. Updated relevant AST nodes to use this stricter parsing function, ensuring better validation and error handling for empty input cases.
These tests validate that the `json_query!` macro correctly rejects various malformed JSONPath expressions. The added cases cover scenarios like empty segments, unexpected tokens, and improper syntax for selectors.
# Conflicts: # src/parser.rs
This comment was marked as resolved.
This comment was marked as resolved.
Thank you for the pr. I will hapily take a look but closer to the weekend (too much stuff now) |
No problem, it's pretty huge |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, can you complement the description in the general readme with the changes:
- what feature means
- how to use it
as soon as the pr is ready to merge.
jp_query = {root ~ segments} | ||
segments = {(S ~ segment)*} | ||
segments = !{(S ~ segment)*} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as far as i get, !
will exclude it from ast as a sep node and include into the parent node instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm gonna quote the pest docs because I'm not the best at explaining and I'm relatively new to pest:
Compound-atomic ($)
Compound-atomic are identical to atomic rules with the exception that rules called by them are not forbidden from generating token pairs.Non-atomic (!)
Non-atomic are identical to normal rules with the exception that they stop the cascading effect of atomic and compound-atomic rules.
So all the not-atomic
does in this particular case is override the compound atomic for the segment rule and all downstream rules, allowing for everything except for the main rule to not be atomic. The main rule being atomic makes it so that leading and trailing white space on the main rule will be rejected by the parser, allowing us to remove the validation logic in parser and allowing the FromPest impl to be more resilient. So adding the !
to the child rules allows them to have leading/trailing white space again.
I guess I should have also added !
to the root but in this case root does not parse to rules so it doesn't make a difference functionally.
First of all, I have to say, this is impressive work. Thank you.
I dont think if you need to eliminate syn unless you see the specific reasons for that :)
At first glance, it looks good, though the tests now takes around 25-30 seconds .
I don't think it is a problem, just need to point it out explicitly in the docs.
precisely
Don't think, it is a problem, but worth noting in docs
it is currently a problem in the current grammar as well, so also fine.
I would suggest to keep the existing format.
I would think about
I would say, statically build is fine |
Thank you for the complement and for taking the time! This is actually the first open source rust repo I've interacted with so thank you for your patience with my lack of experience 😄
That's fair, I've heard complaints of long compile times with syn being a dependency so I figured I'd offer the option, though we do only need one or two of the features syn uses and it's aggressively feature gated so maybe it's fine. Either way the 25 sec test is a much greater priority 😅
Yeah... I didn't even think about that I was just relieved I had finally got them all passing when I was making this. I'm looking into just having the test read them all, and concatenate both all the tests and all the
I will make subsequent commits for docs pointing out these limitations.
Sounds good, I can add these in a subsequent PR that's geared more towards "time to actually use this stuff" or I could put it here, whatever you'd like |
…erals so that rust str->lit parsing doesn't crash on extraneous spaces.
Refactor compile tests by consolidating failing cases into a single `compile_but_expect_err.rs` file and updating test structure. Removed redundant individual files to streamline file management while retaining all test case coverage. Also adjusted parsing utilities to better handle edge cases and ensure clarity.
tests now take 4.5 sec to both build and run. Moving onto docs soon(possibly tomorrow). |
Here's the big one, lots of design choices to be made. I tried to keep everything as simple as possible for now but in the future I could definitely remove a couple hundred lines(ToTokens impls mostly) with declarative macros and once it stabilizes and has a full test suite I can work on better error reporting from the macro side and the FromPest side
I have tried to list out all the options that I have thought of, as well as the one that I think is best but I'm certainly not going to make those choices without presenting them to you and ensuring you agree. These are just my suggestions and I'm happy to expand on my thought process but ultimately it's your choice of course.
In this PR I have not integrated anything I created with the main functionality of this repository because there are still design choices to make, because its a 2600 line addition, and because it's barely tested.
Summary:
jsonpath-ast
json_path!
macro will not suffer longer build timesPestIgnoredPunctuated
,PestLiteralWithoutRule
) for when pest checks that punctuation exist but doesn't parse it, so syn still needs to check that it exists, and FromPest needs to ignore it(because it already guaranteed it) during pest::Rules -> AST conversionMain
writes out "Main::new(....inners)"json_path!
proc macro, depends on jsonpath-ast with feature "compiled-path" enabledMacro Limitations:
r"# #"
strings so in my opinion this is no impact, we can provide a sed regex?json_query!( $["☺"] )
)Macro Output:
From<JPQueryAST> for JPQuery
for all existing structs then just have the macro call.into()
Testing: